0 



Europaisches Patentamt 
European Patent Office 
Office europeen des brevets 



© Publication number: 



0 521 529 A2 



© 



EUROPEAN PATENT APPLICATION 



© Application number: 92111414.6 
© Date of filing: 06.07.92 



© Int. CI. 5 : H04N .7/13 



© Priority: 04.07.91 JP 164422/91 

© Date of publication of application: 
07.01.93 Bulletin 93/01 

© Designated Contracting States: 
DE FR GB 

© Applicant: FUJITSU LIMITED 

1015, Kamikodanaka Nakahara-ku 
Kawasaki-shi Kanagawa 211(JP) 

© Inventor: Hamano, Takashi, c/o Fujitsu 
Limited 



1015, Kamikodanaka, Nakahara-ku 
kawasaki-shi, Kanagawa, 211 (JP) 
Inventor: Sakai, Kiyoshi, c/o Fujitsu Limited 
1015, Kamikodanaka, Nakahara-ku 
Kawasaki-shi, Kanagawa, 2il(JP) 
Inventor: Matsuda, Kiichi, c/o Fujitsu Limited 
1015, Kamikodanaka, Nakahara-ku 
Kawasaki-shi, Kanagawa, 211(JP) 



© Representative: Lehn, Werner, Dipl.-lng. et al 
Hoffmann, Eitle & Partner Patentanwalte 
Arabellastrasse 4 
W-8000 Munchen 81 (DE) 



© An image encoding and transmitting system. 



CNJ 
< 



CM 

w 



© An image encoding and transmitting system is 
an image transmission system for transmitting digital 
data such as TV telephones. The system for dividing 
input frame data into a plurality of blocks before 
transmitting them comprises on its sending side an 
intra-frame/inter-frame determiner (22) for receiving 
intra-frame data and inter-frame data and selecting 
either of them for output, and a difference collator 
(23) for having the intra-frame/inter-frame determiner 
(22) compulsorily output intra-frame data at the time 
of data input for the next frame when one or more 
data in each block forming inter-frame data are larg- 
er than a predetermined threshold. The system also 
comprises on its receiving side an picture element 
domain frame data storage (24) for storing received 
data in the picture element domain, and a motion 
vector detector (25). for detecting according to the 
contents in the storage a motion vector using the 
data in the block surrounding the discarded block in 
order to guess the data in the discarded block. 
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Background of the Invention 

The present invention relates to an efficient 
image data encoding system used in a digital 
transmission of TV telephone and TV conference 
data by an NTSC method, or in a digital transmis- 
sion of high-definition TV (HDTV) data, and more 
specifically to an image encoding and transmitting 
system used in an image transmission system for 
dividing input frame image data into a plurality of 
blocks comprising a plurality of picture elements, 
encoding them in block units, and transmitting the 
encoded data in cell units. 

An asynchronous transfer mode (ATM) has 
been widely accepted and is being standardized to 
realize a broadband ISDN, a new generation net- 
work. In the ATM, data are put into a single format 
"cell" (a fixed length packet) and transmitted by a 
label multiplexing technology. In an image commu- 
nication field, it is regarded as an efficient improve- 
ment in the utilization of lines and regenerated 
image quality through a variable bit rate transmis- 
sion. However, in an efficient encoding system 
based on an inter-frame encoding process, the 
quality of images are badly deteriorated if a cell- 
discarding operation makes an error in image data 
during transmission. Therefore, an efficient encod- 
ing system having a cell-discard compensation 
function is earnestly required. 

Recently, various image data processors have 
been digital oriented, and new systems for trans- 
mitting digital image data signals have been exten- 
sively developed in many application fields such as 
TV telephones, etc. Generally, image data involve a 
large volume of information when transmitted. For 
example, when TV data are digitalized, they require 
100Mb/s transmission capacity and involve 1,500 
times as much as voice data. In the TV broadcast, 
a screen is switched 30 times in a second, and a 
frame is switched every 1/30 second. Normally, 
picture elements are arranged in a two-dimensional 
array in one screen corresponding to one frame. 
Therefore, there have to be a large volume of 
information for image data of one frame even 
though the density of one picture element can be 
represented by 8 bits. 

However, the contents of an image in each 
frame may not vary greatly even when a frame is 
switched every 1.30 second. That is, a large part of 
an image is static including the sky and a scene in 
the background. Only a small part of it varies in 
each frame. Therefore, an inter-frame encoding 
system has been developed to compress and en- 
code image data. 

In the inter-frame encoding system, for exam- 
ple, the difference in density data is obtained be- 
tween corresponding image pictures in two time- 
series frames. The difference is encoded by a 



Huffman code, for example, and transmitted to the 
transmission line. If the density does not vary for 
most picture elements, the difference between pic- 
ture elements results in "0" and can be repre- 

5 sented by about 2 bits. A large difference between 
picture elements requires 8 or more bits. However, 
since only large differences must be encoded, total 
transmission data can be greatly compressed. By 
contrast, a system of encoding data in 1 frame "as 

10 is" is referred to as an "intra-frame encoding sys- 
tem." 

In the inter- and intra-frame encoding systems, 
two basic operations "sampling" and "quantization" 
are required for representing image signals digital- 
75 ly. There are two methods of sampling an image. 
First, an image is sampled by representing by 
density values for a. discrete point array corre- 
sponding to picture elements. Second, a density 
value function (image function) defined on the XY 

20 plane corresponding to a frame plane is expanded 
orthogonally, and the expansion coefficient is deter- 
mined to be a sampling value. 

Figure 1 is a block diagram for explaining the 
general configuration of the image transmission 

25 system using an orthogonal transform. In Figure 1, 
noises are removed through a filter from inputted 
image data by a preprocessor 1 , and picture ele- 
ment data are divided in block units. The block- 
division does not mean an orthogonal transform 

30 collectively performed on data comprising KxN pic- 
ture elements in a two-dimensional array in one 
frame, but is performed to shorten the transforma- 
tion time by obtaining a transformation coefficient 
for each block comprising 4x4 to 16x16 picture 

35 elements. 

In a source encoder 2, data are quantized for 
the reduction of bits after the inter-frame encoding, 
the transformation-encoding, etc. as encoding pro- 
cesses for removing redundancy in space and time 

40 directions. The orthogonal transforms performed in 
this technology are, for example, Adamar trans- 
form, cosine transform. Karounen Roebe transform, 
etc. Among them, the discrete representation in 
cosine transforms is commonly used, and the dis- 

45 crete cosine transform (DCT) is applied most. 

In such orthogonal transformation-encoding 
systems, the inter-relation in a small area is re- 
garded and an orthogonal transform is performed 
using picture elements of a small area as a numeri- 

50 cal string. The resultant transformation coefficient 
corresponds to a frequency component indicating 
each level ranging from a low frequency to a high 
frequency. Since an image signal normally has a 
low frequency component rather than a high fre- 

55 quency component, a larger number of bits are 
assigned to a low frequency component and a 
smaller number of bits to a high frequency compo- 
nent prior to the quantization. Thus, the volume of 
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transmission data can be reduced greatly. 

Figure 1 shows a variable length encoder 3 for 
encoding data by a Huffman code, run-length code, 
etc. to eliminate statistical redundancy, and a cell 
assembler 4 for assigning side information for de- 
tecting cell-discard and a header to a cell. Thus, an 
assembled cell is outputted through a network. 

On the receiving side, a cell disassembler 5 
detects cell-discard in a cell inputted through the 
network, and the cell is disassembled to data. 
Then, the received bit string is decoded by a 
variable length decoder 6, and image data are 
decoded by a source decoder 7 by performing the 
source encoding process in reverse. Then, a post- 
processor 8 performs an unblock process, a noise 
removal process using a filter, etc., and the data 
are outputted. 

Figure 2 is a block diagram for explaining the 
configuration for the inter-frame DCT encoding sys- 
tem as the first example of a typical conventional 
encoding system. In Figure 2, a subtracter 1 1 on 
the sending side calculates the difference between 
inputted image data divided into block data and the 
data obtained by multiplying the block data stored 
in a frame memory 9 at the same block position 
but in the preceding frame by the leak coefficient a 
provided by a leak coefficient unit 10. The dif- 
ference and the inputted data are applied to an 
intra-frame/inter-frame determiner 12 for determin- 
ing which is effective, to encode the data in the 
present frame, that is, to encode the intra-frame 
data as is, or to encode the difference between the 
data in the present frame and those at the cor- 
responding position in one frame before the 
present data. The data determined to be more 
effective are applied to a discrete cosine transform 
(DCT) unit 13, and quantized by a quantizer 14. 

The output of the quantizer 14 and the intra- 
frame/inter-frame determination result indicated by 
the broken line shown in Figure 2 are outputted to 
the variable length encoder 3 shown in Figure 1. 
Simultaneously, the output of the quantizer 14 is 
inversely transformed again to the picture element 
domain by an inverse discrete cosine transform 
(IDCT) unit 15, added by an adder 16 to the 
product obtained by multiplying the data stored in 
the frame memory 9 by a from the leak coefficient 
unit 10, and then stored again in the frame memory 
9. However, when an intra-frame data encoding is 
selected, the adder 16 does not add the data in the 
prior frame according to the input of a determina- 
tion signal as shown by the broken line, but stores 
the output of the IDCT unit 15 as is in the frame 
memory 9. 

In Figure 2, on the receiving side, the input 
data are inversely transformed to the picture ele- 
ment domain by an IDCT unit 17. The result is 
added by an adder 20 to the data obtained by 



multiplying the data in the preceding frame stored 
in a frame memory 18 by # from a leak coefficient 
unit 19, and then outputted to the post-processor 8 
shown in Figure 1 and stored again in the frame 

5 memory 18. When an intra-frame data encoding is 
selected, the data in the prior frame are not added. 
For a block which does not receive data due to 
cell- discard, the prior frame data is added to the 
input 0, and the result is stored in the frame 

10 memory 18. That is. at the cell-discard, the. prior 
frame data are stored as is to the frame memory 
18. 

In Figure 2, both on the sending and the re- 
ceiving sides, the leak coefficient unit exists be- 

is tween the output of the frame memory and the 
adder. The product obtained by multiplying the 
output of the frame memory by the leak coefficient 
a is used as a value of the data in the preceding 
frame. If the leak coefficient a is set to a value 

20 between 0 and 1 and an error E arises between the 
contents stored in the frame memories on both the 
sending and the receiving sides, the error results in 
Ex a n after n frames. The error gradually gets 
smaller with time, and finally converges to zero. 

25 Therefore, the quality of a regenerated image can 
be restored when the error value has decreased 
down to near zero even though an error E has 
arisen on the sending and the receiving sides due 
to cell-discard and has incurred the deterioration in 

30 the quality of a regenerated image. 

Figure 3 is a block diagram for explaining the 
configuration for realizing the DCT inter-frame en- 
coding system as the second embodiment of the 
conventional encoding system. The configuration of 

35 this system is similar to the system shown in 
Figure 2 except the following points: the discrete 
cosine transform (DCT) unit 13 is located in the 
source encoder 2, the input image data are trans- 
formed by the DCT unit 13 to a frequency domain 

40 coefficient, and the inter-frame or intra-frame data 
encoding system is selected for the transformed 
coefficient data. 

On the receiving side, data in the frequency 
domain are decoded using the frame memory 18 

45 depending on the selection of the inter-frame or 
intra-frame encoding system. The result is trans- 
formed by the IDCT unit 17 to the picture element 
domain, and outputted to the post-processor 8. 

Figure 5 is a configurational block diagram for 

so explaining the motion compensation inter-frame 
DCT encoding system of the prior art technology. It 
is similar in its configuration to Figure 2 showing 
the first example of the prior art technology. How- 
ever, since the intra-frame encoding system is not 

55 used in the operation shown in Figure 5, the intra- 
frame/inter-frame determiner 12 is not provided on 
the sending side. Instead, a motion vector detector 
100 for detecting a motion vector of each block 
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according to inputted data and the data in the 
preceding frame stored in the frame memory 9 and 
a variable delay unit 101 for delaying the contents 
stored in the frame memory 9 and outputting them 
to the leak coefficient unit 10 are added to the ; 
sending side. 

Also on the receiving side, a variable delay unit 
102 for delaying by the necessary time period the 
data in the preceding frame stored in the frame 
memory 18 according to a motion vector received 7i 
from the sending side and outputting them to the 
leak coefficient unit 19 is added. 

In the conventional encoding system shown in 
Figures 2 and 3, the image quality can be auto- 
matically restored with time using a leak coefficient /< 
a even though the quality of an image has deterio- 
rated due to cell-discard. For a block in which the 
time difference is comparatively small between the 
previous frame and the present frame, an error in 
the image data between the sender and the re- 2c 
ceiver is small, and can rapidly converge to zero 
using the leak coefficient. Therefore, the deteriora- 
tion of the image quality can be limited to a visu- 
ally admitted level. However, for a block in which 
the time difference is comparatively large, the error 25 
between these image data becomes large and the 
deterioration in the image quality is badly outstand- 
ing when the cell-discard rate is high because the 
error takes much time in converging using a leak 
coefficient. 30 

Summary of the Invention 

An object of the present invention is to shorten 
the time taken for restoring the deteriorated quality 35 
of an image, that is, for converging to zero the 
error in the image data between a sender and a 
receiver caused by cell-discard. Another object of 
the present invention is, when cell-discard has aris- 
en, to minimize an error in an image data between 40 
a sender and a receiver by guessing using a mo- 
tion vector a data block in which the cell-discard 
has arisen. 

A feature of the present invention relates in an 
image encoding and transmitting system for divid- 45 
ing inputted frame image data into a plurality of 
blocks comprising picture elements, encoding them 
in block units, and transmitting said encoded data 
in packet units, said system comprising on its 
sending side, intra-frame' inter-frame determining 50 
means for receiving the first difference data as the 
difference between inputted frame data and data in 
the preceding frame or the difference between in- 
putted frame data and data obtained by multiplying 
said data in the preceding frame by a constant a, 55 
and selecting and outputting either of the intra- 
frame data as the inputted frame data or the inter- 
frame data as the first difference data, and dif- 



ference collating means for receiving inputted 
frame data and the second difference data between 
said inputted frame data and the data in one frame 
before the present frame, and having said inter- 
frame/intra-frame determiner compulsorily output 
the intra-frame data at the data input of the next 
frame for the same block position when fore than 
one piece of data in each block forming the second 
difference data are larger than a predetermined 
threshold, wherein the deterioration of image qual- 
ity of frame data having a large difference can be 
prevented. 

Brief Description of the Drawings 

One of skill in the art can easily understand 
additional features and objects of this invention 
from the description of the preferred embodiments 
and some of the attached drawings. In the draw- 
ings: 

Figure 1 is a block diagram for explaining the 
general configuration of the image transmission 
system; 

Figure 2 is a block diagram for explaining the 
configuration of the first embodiment of the 
inter-frame DCT encoding system; 
Figure 3 is a block diagram for explaining the 
configuration of the second embodiment of the 
DCT inter-frame encoding system; 
Figure 4 is a block diagram for explaining the 
configuration of the motion compensation inter- 
frame DCT encoding system according to the 
third example of the prior art technology. 
Figure 5 is a block diagram for explaining the 
principle of the present invention; 
Figured 6 shows the concept of the compulsory 
intra-frame encoding system; 
Figure 7 shows an error in the compulsory inter- 
frame encoding system; 

Figure 8 shows how to convert picture element 
data to a frequency domain coefficient. 
Figure 9 is a block diagram for explaining the 
configuration of the first principle of the present 
invention; 

Figure 10 shows the detailed configuration of 
the intra-frame/inter-frame determiner, the dif- 
ference collator, and the one-frame delay unit 
shown in Figure 9; 

Figure 11 shows the concept of the motion 
vector detecting system in the second principle 
(at a normal reception); 

Figure 12 shows the concept of the motion 
vector detecting system in the second principle 
(of the prior art technology); 
Figure 13 shows the concept of the motion 
vector detecting system in the second principle 
(of the present invention); 

Figure 14 is the block diagram for explaining the 
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configuration of an embodiment of the second 
principle; 

Figure 15 is a block diagram for explaining the 
configuration of other embodiments of the sec- 
ond principle; 

Figure 16 shows an embodiment of a cell for- 
mat. 

Figure 17 is a block diagram for explaining the 
configuration of the cell disassembler. 
Figure 18 is a block diagram for explaining the 
configuration of the adder. 

Figure 19 is a block diagram for explaining the 
configuration of the teak coefficient unit. 
Figure 20 is a flowchart of an embodiment of a 
data replacing process using a motion vector; 
Figure 21 is a flowchart of an embodiment of the 
first motion vector detecting process; 
Figure 22 is a detailed flowchart of a motion 
vector detecting process shown in Figure 21; 
Figure 23 is a flowchart of an embodiment of the 
second motion vector detecting process; and 
Figure 24 is a detailed flowchart of a motion 
vector detecting process shown in Figure 23. 
Figure 25 is a block diagram for explaining the 
configuration of the MC replacement unit shown 
in Figures 14 and 15. 

Figure 26 is a block diagram for explaining the 
configuration of the embodiment to which a part 
of the second principle is applied. 
Figure 27 is a block diagram for explaining the 
configuration of the MC replacement unit shown 
in Figure 26. 

Figure 28 is a flowchart for explaining the cal- 
culation for obtaining an average value of motion 
vectors shown in Figure 26. 

Description of the Preferred Embodiment 

Figure 5 is a block diagram for explaining the 
principle of the present invention, it shows the 
principle of the image encoding and transmitting 
system used in an image transmission system in 
which image data in an inputted frame are divided 
into blocks each comprising a plurality of picture 
elements, the data are encoded in block units, and 
the encoded data are transmitted in cell units. 

For the detailed explanation of the present in- 
vention, a transmission method in which encoded 
data are transmitted in an ATM cell, for example. 
However, an encoded data transmission method is 
not limited to the method using the ATM cell, but 
can be realized using any kind of packet such as 
variable length packets, fixed length packets, etc. 

Figure 5 is a block diagram for explaining the 
first principle. It shows the principle of the image 
encoding and transmitting system for minimizing 
the influence of cell-discard by performing a com- 
pulsory intra-frame encoding on the frame data 



after the data having a comparatively large dif- 
ference are transmitted through an inter-frame en- 
coding so as to prevent the image quality from 
being greatly deteriorated when data having a large 

5 difference are cell-discarded. 

In Figure 5A, an intra-frame-inter-frame deter- 
miner 22 receives inputted frame data and the first 
difference data which indicate the difference be- 
tween the inputted frame data and the data in the 

ro preceding frame or between the inputted frame 
data and the data obtained by multiplying the data 
in the preceding frame by a constant, for example, 
a leak coefficient a. and selects and outputs either 
of the intra-frame data as the inputted frame data 

;s or the inter-frame data as the first difference data. 
The selection is determined by calculating a sum 
of squares of data in each block, collating the sums, 
of squares for an intra-frame data and an inter- 
frame data, and selecting and outputting whichever 

20 has a smaller sum of squares. Thus, the length of 
the data code transmitted to the receiving side can 
be shortened. 

A difference collator 23 receives the second 
difference data between the inputted frame data 

25 and the data in one frame before the present 
frame, and causes the inter-frame/intra-frame deter- 
miner 22 to compulsorily output the intra-frame 
data when the data at the same block position in 
the next frame are inputted when more than one 

30 piece of data in each block forming the difference 
data is larger than a predetermined threshold. Al- 
though comparatively large difference data are 
inter-frame -encoded, transmitted, and then cell- 
discarded, the quality of the image can be pre- 

35 vented from being deteriorated by intra-frame-en- 
coding the next frame data. The i nter- frame/i ntra- 
frame determiner 22 and the difference collator 23 
are provided on the sending side in an image 
transmission system. 

40 Figure 5B is a block diagram for explaining the 

principle of the second embodiment. It shows the 
principle of the image encoding and transmitting 
system for guessing data in a block whose data 
were not received due to cell-discard by detecting 

45 a motion vector using the data in the blocks sur- 
rounding the missing block. 

In Figure 5B, a picture element domain frame 
data storage 24 is. for example, a frame memory, 
and stores received image data in a picture ele- 

50 ment domain. If image data are transmitted as a 
coefficient of a frequency domain orthogonally 
transformed in the transmission line, they are 
stored in the picture element domain frame data 
storage 24. 

55 A motion vector detector 25 detects a motion 

vector using the data in the blocks surrounding the 
block whose encoded data were not received due 
to cell-discard among the frame data stored in the 
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picture element domain frame data storage 24 
when a cell-discard detection signal is inputted. 
The motion vector detector 25 replaces data using 
this motion vector. The picture element domain 
frame data storage 24 and the motion vector detec- 
tor 25 are provided on the receiving side in an 
image transmission system. 

As shown in Figure 5A for explaining the first 
principle, an object of the present invention is to 
shorten the time taken for converging to zero an 
error caused by cell-discard in image data between 
a sender and a receiver, that is, to shorten the time 
taken for reconstructing discarded data in a block 
having a large inter-frame difference. If the inter- 
frame encoding system is performed in spite of a 
comparatively large inter-frame difference, the data 
at the same block position in the next frame are 
compulsorily intra-frame-encoded. Therefore, trie 
next frame can be encoded without the influence of 
the preceding frame even though a large difference 
block data may be discarded, thereby greatly shor- 
tening the time taken for restoring acceptable im- 
age quality. 

As shown in Figure 5B for explaining the sec- 
ond principle of the present invention, a motion 
vector is detected when a cell-discard detecting 
signal is inputted to the receiving side so that 
determination can be made as to which position of 
a block in the preceding frame replaces the dis- 
carded block. Thus, data are replaced using the 
motion vector. However, a motion vector cannot be 
obtained directly from the discarded block, but can 
be obtained indirectly using a surrounding block 
not affected by the cell-discard. 

In the first system of obtaining a motion vector 
for replacing block data, motion vectors each for 
one or more blocks not affected by cell-discard are 
obtained. Then, an average value of the motion 
vectors is calculated as a replacing motion vector. 
In the second system, a general motion vector is 
detected in one block group comprising one or 
more blocks not affected by cell-discard. It is used 
as a replacing motion vector. 

In Figure 5B for explaining the second princi- 
ple, the picture element domain frame data storage 
24 stores received image data in the picture ele- 
ment domain. Among the frame data stored here, 
block data not received due to cell-discard are 
replaced by the motion vector detector 25. If en- 
coded data in the coefficient domain orthogonally 
transformed in the transmission line are transmit- 
ted, a second frame memory for storing the frame 
data in the coefficient domain as well as a frame 
memory corresponding to the picture element do- 
main frame data storage 24 are provided on the 
receiving side. When cell-discard has arisen, block 
data in the preceding frame stored in the second 
frame memory are inversely transformed to the 



picture element domain. Then, the inversely-trans- 
formed data are stored in a frame memory cor- 
responding to the picture element domain frame 
data storage 24. 

s As described above, when data having a com- 

paratively large inter-frame difference are transmit- 
ted, an intra-frame encoding is compulsorily se- 
lected for the same block position in the next 
frame, and. on the receiving side, the data in the 

w discarded block can be replaced using a motion 
vector for the blocks surrounding the discarded 
block. 

Figure, 6 shows the concept of the first princi- 
ple, that is, the compulsory intra-frame encoding 
rs system. Figure 7 indicates the prior art technology 
in which the difference between the present frame 

, . and the preceding frame are transmitted even 
though the difference is comparatively large unless 
an intra-frame encoding is selected from between 

20 the intra-frame encoding and the inter-frame en- 
coding. Therefore, the time taken for reconstructing 
the deteriorated image quality caused by cell-dis- 
card is kept until, at the maximum, an error con- 
verges to zero using a leak coefficient. 

25 In the concept of the present invention shown 

in Figure 6, an inter-frame coding is selected, for 
example. If the difference is comparatively large, a 
compulsory intra-frame-encoding is selected for the 
next frame. Therefore, the error in the next frame is 

30 immediately converged to zero even though data 
having a large difference are discarded. 

Figure 7 is a view for explaining an error in the 
compulsory intra-frame encoding system. Figure 
7A indicates the prior art technology in which an 

35 error converges based on an almost equal time 
constant regardless of an error level. Inter-frame 
data having a large difference are discarded. When 
an error is very large, it takes a long time for it to 
converge to zero. By contrast, in the present inven- 

40 tion shown in Figure 7B, an error can converge in 
almost the same time as the conventional system if 
the difference is smaller than a predetermined 
threshold and a compulsory intra-frame encoding is 
not selected for the next frame. However, an error 

45 immediately converges to zero when the difference 
is larger than a predetermined threshold and a 
compulsory intra-frame encoding is selected for the 
next frame. 

Figure 8 shows a block diagram for explaining 
so how to convert picture element data to a coefficient 
for a frequency domain by the discrete cosine 
transform. For example, a discrete cosine transform 
is performed on 8x8 picture element data to obtain 
64 transformation coefficients for the frequency do- 
ss main. The coefficients refer to the frequency com- 
ponent, and the coefficient Coo at the upper left 
corner generally indicates a direct current compo- 
nent. The coefficients toward right and downward 
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therefrom show the higher frequency components. 

Figure 9 is a block diagram for explaining the 
configuration of the embodiment of the first princi- 
ple. It shows the configuration on the sending side 
where a discrete cosine transform is performed on 
inputted data, and an intra-frame/inter-frame deter- 
mination is made on the transformed data in the 
coefficient domain like in the second example of 
the encoding system of the prior art technology 
shown in Figure 3. The same parts as those in 
Figure 3 are assigned the same numbers. 

The configuration shown in Figure 9 comprises, 
in addition to the configuration on the sending side 
shown in Figure 3, a subtracter 30 for calculating 
the difference between the output of the DCT unit 
13 and the output of the frame memory 9, that is, 
for calculating an inter-frame difference for all data 
in a block, a difference collator 31 for determining 
whether or not the difference is larger than a pre- 
determined threshold, a one-frame delay unit 32 for 
delaying by one frame the output of the difference 
collator 31 and having the intra-frame/inter-frame 
determiner 12 select a compulsory intra-frame en- 
coding for the next frame when the difference 
collator 31 determines that the difference is larger 
than the threshold. 

Figure 10 is a detailed block diagram of the 
intra-frame/inter-frame determiner 12, the differ- 
ence collator 31, and the one-frame delay unit 32 
shown in Figure 9. In Figure 10, the difference 
collator 31 corresponds to comparators 33a - 33n 
and an AND circuit 34, and the one-frame delay 
unit 32 corresponds to a first-in-first-out memory 
(FIFO) 35. 

The number of comparators 33a - 33n is equal 
to the number of the transformation coefficient data 
in a block. In each comparator, an inter-frame dif- 
ference of block data is compared with a threshold. 
If the absolute value of the difference is larger than 
the threshold, "0" is outputted. If it is smaller than 
the threshold, "1" is outputted. If the inter-frame 
differences for all transformation coefficient data in 
a block are smaller than the threshold, the AND 
circuit 34 outputs "1". If any one absolute value of 
the difference in any data is larger than the thresh- 
old, it outputs "0", and the value is stored in the 
FIFO 35. 

The intra-frame/inter-frame determiner 12 
shown in Figure 5 can be formed of a sum-of- 
squares calculator 36a for obtaining a sum of 
squares of data in transformation coefficient data in 
a block for the inter-frame difference data outputted 
by the subtracter shown in Figure 9, a sum-of- 
squares calculator 36b for obtaining a sum of 
squares of data in transformation coefficient data in 
a block for the intra-frame data outputted by the 
DCT unit 13, a comparator 37 for comparing the 
outputs of two sum-of-squares calculators, an AND 



circuit 38, a delay unit 39a for delaying the output 
of the subtracter 1 1 by the time taken for intra- 
frame/inter-frame determination, a delay unit 39b 
for delaying the output of the DCT unit 13 by the 

5 above described time, and a selector 40 for output- 
ting the output from either of two delay units, each 
being shown in Figure 10. 

In Figure 10, the comparator 37 outputs "0" 
when a sum of squares of an inter-frame difference 

w data is larger than a sum of squares of an intra- 
frame data, and outputs "1" when a sum of 
squares of an inter-frame difference data is smaller 
than a sum of squares of an intra-frame data. When 
the comparator 37 outputs ,f 0'\ the AND circuit 38 

75 outputs "0". At this time, the selector 40 outputs 
the output of the delay unit 39b, that is, intra-frame 
data, to the variable length encoder 3 shown in 
Figure 22. 

By contrast, an inter-frame encoding may be 

20 selected when the comparator 37 outputs "1". In 
this case, the FIFO 35 must also output w 1". That 
is, the difference in all data in a block between the 
present and the previous frames must be smaller 
than a predetermined threshold. Then, the AND 

25 circuit 38 outputs "1", the selector 40 outputs the 
output from the delay unit 39a, that is, the inter- 
frame difference data outputted by the subtracter 
1 1 , to the variable length encoder 3. If the FIFO 35 
outputs "0" while the comparator 37 outputs n 1", 

30 that is, if the inter-frame difference at the same 
block position between the present and previous 
frames is larger than a predetermined threshold, 
the AND circuit 38 outputs "0", thereby selecting a 
compulsory intra-frame encoding. 

as Figures 11-13 show the concept of the motion 

vector detecting system of the second principle of 
the present invention. Figure 1 1 shows an example 
of a present frame, one frame before the present 
frame, and the next frame when no cell-discard has 

40 arisen, that is, transmission data are normally re- 
ceived. In this example, an image is positioned at 
the right, and a camera is moved to the left. 

Figure 12 shows an example of the conven- 
tional system in which data are replaced with block 

4$ data in a preceding frame when cell-discard has 
arisen. In Figure 12, if block 10 is discarded in the 
present frame, the missing data must be normally 
replaced with those in block 9 in the preceding 
frame. However, since the data in block 9 in the 

so preceding frame is also discarded, the data is 
replaced with those in block 10 in the preceding 
frame "as is". In the next frame, the data in block 9 
in the present frame, that is, the data with only its 
upper left corner lacking, are received unless addi- 

55 tional cell-discard does not arise. However, the cell- 
discard affects block 10 in the present frame, 
thereby generating a block data with its upper right 
corner lacking. 
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Figure 13 shows an example of replacement 
through a motion vector. Like in Figure 12, the data 
in block 10 are assumed to be discarded in the 
present frame. As shown in Figure 13B, if, for 
example, a motion vector is detected in block 4 
above block 10 and in block 16 below block 10, 
and block 3 and 15 in the preceding frame are 
determined to move to block 4 and 16 respectively, 
then block 9 is assumed to move to block 10. 
Thus, as shown in Figure 15, the influence of the 
cell-discard in block 10 can be eliminated by re- 
placing the data in block 10 in the preceding frame 
with those in block 9. 

Figure 14 is a block diagram for explaining the 
configuration of an embodiment of the second prin- 
ciple. It shows the configuration on the receiving 
side for performing by the IDCT unit 17 an inverse 
orthogonal transform on received input data and 
then decoding them like in the first example of the 
prior art technology shown in Figure 2. In Figure 
14, like the frame memory 18 shown in Figure 2, a 
frame memory 41 is used in a common decoding 
process, and also used for replacing discarded 
block data using a motion vector. A motion com- 
pensation replacement unit 42 for replacing dis- 
carded block data using a motion vector detects a 
motion vector using block data surrounding the 
discarded block data when a cell-discard detection 
signal is inputted by the cell disassembler 5 and 
replaces the discarded block data. Detection of a 
motion vector and replacement of block data are 
described later. 

Figure 15 is a block diagram for explaining the 
configuration of another embodiment of the second 
principle of the present invention. It corresponds to 
the second prior art technology shown in Figure 3. 
The frame memory 18 stores the data in the coeffi- 
cient domains. A motion vector cannot be detected 
in these data. Therefore, the IDCT unit 17 stores 
the data converted to the picture element domain 
in a second frame memory 43. When cell-discard 
is detected, an MC replacement unit 44 replaces 
discarded block data with the block data stored in 
the second frame memory 43. When cell-discard is 
detected, a detection signal is applied to the leak 
coefficient unit 19. The data stored in the frame 
memory 18 are not processed with a leak coeffi- 
cient, but stored again in the frame memory 18. 
Simultaneously, they are transformed to the data in 
the picture element domain by the IDCT unit 17„ 
and then outputted to the second frame memory 
43. 

In the embodiment of the second principle of 
the present invention, the operation when an intra- 
frame encoding is selected on the sending side is 
the same as the conventional operation shown in 
Figures 2 and 3. 

Next, a cell-discard detecting system in the 



second principle of the present invention is de- 
scribed below. Figure 22 shows an embodiment of 
a cell format. In Figure 22, a 5-byte header is 
stored before the 53-byte cell, followed by a 1-byte 

5 sequence number as side information. Cell-discard 
can be detected by monitoring a sequence num- 
ber. The encoded data in a cell occupy 47 bytes. 

Figure 17 is a configurational block diagram of 
the celt disassembler 5. In Figure 23, the header is 

w removed from the received cell by a header remov- 
ing unit 103. The cell whose header is removed is 
applied to a disassembler 104, and is divided to a 
1-byte sequence number and 47-byte encoded 
data. The value in a cell counter 105 is stepped up 

75 when a cell is received. The counter value and the 
sequence number outputted from the disassembler 
104 are compared by a comparator 106, and a cell- 
discard detection signal is outputted when a se- 
quence number is omitted. The 47-byte encoded 

20 data from the disassembler 104 is provided for the 
variable length decoder 6 shown in Figure 1 as 
encoded data by a read controller 108 through a 
memory 107. When cell-discard is detected, "0", 
for example, are outputted for 47 bytes as dummy 

25 data. 

Figure 18 is a configurational block diagram of 
the adder 20 shown in Figures 14 and 15. When an 
intra-frame encoding is selected, inputted data and 
the encoded data from a read controller 108 shown 
30 in Figure 17 and "0" are added together and 
outputted. When an inter-frame encoding is se- 
lected, an output from the leak coefficient unit 19 
and the encoded data are added together and 
outputted. 

35 Figure 19 is a configurational block diagram of 

the leak coefficient unit 19. For example, when cell- 
discard is detected as shown in Figure 21, dummy 
data "0" are applied from the cell disassembler. At 
this time, the data in the preceding frame stored in 

40 the frame memory 18 are outputted as is to the 
adder 20 without performing a multiplication by the 
leak coefficient. When cell-discard is not detected, 
the content in the frame memory 18 is multiplied 
.by the leak coefficient , and the result is outputted. 

45 Figure 20 is a flowchart of an embodiment of a 

data replacement process using a motion vector. In 
Figure 20, when the process starts, determination 
is made in step S 51 as to whether or not block B- 
(i,j) has been discarded. In block B(i,j), the coordi- 

50 nates of the point at the upper left corner are (i,j) 
as shown in Figure 20B. One block comprises 8x8 
picture elements. 

In step S 52, the difference y of the coordi- 
nates in the vertical direction (e.g. downward) from 

55 the upper left corner point (i.j) of a block is set to 
W 0 M . In step S 53, the difference x of the coordi- 
nates in the horizontal direction (e.g. to the right) is 
set to "0". In step S 54, the data for the coordi- 
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nates (i + x.j + y) in the present frame can be cal- 
culated by the following expression using the mo- 
tion vectors x and y. 

presf(i + x,j + y) = prevf(i + x + vector x.j + y + vector 

y) 

where "presf" indicates the present frame, and 
"prevf" indicates the preceding frame. 

Then, in step'S 55, the value of x is stepped 
up. In step S 56, determination is made as to 
whether or not x equals 8. If not, the processes are 
repeated from step S 54. 

If x equals 8 as the result of the determination 
in step S 56, the difference of the coordinates in 
the horizontal direction, that is. the horizontal re- 
placement range, by a. motion vector, has reached 
the width of one block. Accordingly, in step S 57, 
the value of y is stepped up. In step S 58, deter- 
mination is made as to whether or not the value of 
y has reached 8. If not, the processes are repeated 
from step S 53. In step S 58, if y has reached 8, 
the process is terminated. If block B(i,j) is not 
determined to have been discarded in step S 51, 
the process is terminated immediately. 

Figure 21 is a flowchart for explaining the first 
embodiment of an average value calculation pro- 
cess of a motion vector for one or more blocks in 
the motion vector general detection process. In the 
flowchart shown in Figure 21 A, when the process 
starts, determination is made in step S 60 as to 
whether or not block B(i,j) has been discarded. If 
not, the process is terminated immediately. 

When block B(i,j) is determined to have been 
discarded, the number of significant blocks, that is, 
the number of significant blocks blkct used for 
detecting a motion vector is set to "0" in step S 
61. In step S 62, determination is made as to 
whether or not block B(i,j-8) has been discarded. 
As shown in Figure 21 B, block B(i,j-8) is positioned 
above block B(i,j) which is detected as being dis- 
carded. 

If the block above (shown in Figure 21 B) is not 
discarded in step S 62, a motion vector is detected 
using the block above in step S 63. In step S 64, 
the number of significant blocks blkct is stepped 
up and the vectors x and y are obtained. Then, 
control is transferred to step S 65. If the block 
above is detected as being discarded in step S 62, 
it is impossible to detect a motion vector. There- 
fore, control is transferred to the process in step S 
65 without performing the processes in steps S 63 
and S 64. 

In step S 65, determination is made as to 
whether or not the block below the discarded block 
B(i.j) shown in Figure 21 B has been discarded. If 
not, a motion vector is detected using the block 
below in step S 66 like it is detected using the 



block above in step S 63. In step S 67, the value of 
blkck is stepped up, and the motion vectors x and 
y are obtained using the vector obtained in step S 
66 and the motion vectors x and y obtained in step 
5 S 64. If the block above is discarded and the 
process in step S 64 is not performed, the motion 
vectors x and y obtained in step S 67 equal to the 
motion vectors detected in step S 66. If cell-discard 
is detected in step S 65, control is transferred to 
10 the process in step S 68 without performing the 
processes in steps S 66 and S 67. 

In step S 68. determination is made as to 
whether or not the number of significant blocks 
equals "0". If yes, both the blocks above and 
rs below are discarded. In this case, since a motion 
vector cannot be obtained, the process is termi- 
nated with the motion vectors x and y set to "0" in 
step S 69. If the number of significant blocks does 
not equal "0", the values of the motion vectors x 
20 and y already obtained in step S 70 divided by the 
number of significant blocks respectively are as- 
signed as the final values of the vectors x and y, 
and the process is terminated. 

Figure 22 is a detailed flowchart of a motion 
25 vector detecting process shown in Figure 21. It is 
used in detecting a motion vector for a block above 
described in step S 63 in Figure 21 and used in 
detecting a motion vector for a block below de- 
scribed in step S 66. 
30 In Figure 22, when the process is started, in 

step S 71. the value of sum 1 is set to 10,000 as 
the minimum initial value of the sum of absolute 
values described later. In step S 72, the value of 
the vertically-variable vector "vecy" is set to -4. In 
35 step S 73, the value of the horizontally-variable 
vector "vecx" is set to -4. In step S 74, using the 
variable vectors set in steps S 72 and S 73, a 
difference is calculated between the data of a point 
obtained by adding a variable vector to the coordi- 
40 nates of any point in the preceding frame, and the 
data of the point at the same position as the above 
described "any point" in the present frame. The 
sum of the absolute values sum 2 of the differ- 
ences described above are calculated for all the 
45 points in the block. In step S 75, determination is 
made as to whether or not the grand total of the 
calculation performed in step S 75 is smaller than 
the initial value set in step S 71. 

If the sum is smaller than the initial value in 
50 step S 75. variable vectors are assigned as motion 
vectors in step S 76. In step S 74, the obtained 
sum of absolute values sum 2 is determined to be 
the minimum value sum 1 of the sum of absolute 
values, and control is transferred to step S 77. If 
55 the sum is not smaller than the initial value in step 
S 75, the variable vector is not determined to be 
close to the motion vector. Therefore, control is 
transferred to the process in step S 77 without 
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performing the process in step S 76. 

In step S 77, the value of the horizontally- 
variable vector is stepped up. In step S 78, deter- 
mination is made as to whether or not the value of 
the horizontally-variable vector has reached 5. If 
not, data have not been checked up to the half of 
the width of the block. The processes are repeated 
from the process in step S 74. If the value of the 
horizontally-variable vector has reached 5 in step S 
78, the value of the vertically-variable vector is 
stepped up. In step S 80, determination is made as 
to whether or not the value has reached 5. If not, 
the processes are repeated from the process in 
step S 73. If yes. the process is terminated imme- 
diately. 

Figure 23 is a flowchart for explaining the sec- 
ond embodiment of the motion vector general de- 
tecting process. In the second embodiment, unlike 
the first embodiment described by referring to Fig- 
ure 21, the blocks above and below a discarded 
block are regarded as one group. Among them, 
one motion vector is detected. 

In the flowchart shown in Figure 23A, when the 
process is started, determination is made in step S 

81 as to whether or not B(i,j) has been discarded. If 
not, the process is terminated immediately. If it has 
been discarded, determination is made in step S 

82 whether or not block B(i,j-8), that is, the block 
above (refer to Figure 23B), is discarded. If not. the 
vertical coordinate minj at the upper limit within the 
motion vector detection range is set to j-8 in step S 
83. If yes. the value is set to j in step S 84 and 
control is transferred to the process in step S 85. 

In step S 85, determination is made as to 
whether or not block B(i,j + 8), that is, the block 
below has been discarded. If yes, the vertical co- 
ordinate maxj at the lower limit within the motion 
vector detection range is set to j + 7 in step S 86. If 
not, the value is set to j + 15 in step S 87, and 
control is transferred to the process in step S 88. 

In step S 88, determination is made as to 
whether or not the minimum value minj of the 
vertical coordinate within the detection range is 
equal to j, and the vertical coordinate maxj at the 
lower limit within the range is equal to j+7. If yes, 
both the blocks above and below has been dis- 
carded. Therefore, a motion vector cannot be de- 
tected, and the process is terminated with these 
values set to "0" in step S 89. If even one of the 
above described minj and maxj is different from the 
indicated value, a motion vector is detected in step 
S 90 with blocks having no cell-discard between 
the maximum and minimum values of the vertical 
coordinate established as one group, and then the 
process is terminated. 

Figure 24 is a detailed flowchart for explaining 
the motion vector detecting process shown in Fig- 
ure 23. The flowchart is similar to that shown in 



Figure 22 except in the domain in which a sum of 
absolute values of differences is obtained in step S 
94 and in the expression which is different from 
that in S 74 shown in Figure 22. The first para- 

5 graph of the object in a block S 94 corresponds to 
the sum of absolute values of the differences for 
the blocks above. The second paragraph corre- 
sponds to the sum of absolute values of the differ- 
ences for the blocks below. With these paragraphs, 

io if an upper limit is smaller than a lower limit in a 
vertical coordinate y, the total sum in the paragraph 
equals "0". 

Figure 25 is a configurational block diagram of 
the MC replacement unit shown in Figures 14 and 

;s 15. The embodiment shown in Figures 21 and 22 
are different from that shown in Figures 23 and 24 
in the motion vector detecting system. However, 
they have a similar configuration of the MC re- 
placement unit. 

20 As described in the above described flowchart, 

data in the blocks above and below must be re- 
ceived in processing a target replacement block. 
Thus, as shown in Figure 25, a B(i,j-8) discard 
detector 112 detects the discard of a block above 

25 the target block according to a 2-block line delay 
unit 110, a B(i,j) discard detector 113 detects the 
discard of the target block according to a 1 -block 
line delay unit 111, and a B(i,j + 8) discard detector 
114 detects the discard of a block below the target 

30 block. A motion vector 115 detects a motion vector 
according to the outputs from these 3 discard 
detectors 112 - 114 and the content of the frame 
memory. Using the detected motion vector, a re- 
placement controller 116 reads data in the prevf 

35 domain in the frame memory for storing data in the 
preceding frame, and replaces the discarded data 
with them by writing the replacement data into the 
presf domain. 

Figure 26 is a configurational block diagram for 

40 explaining the embodiment to which a part of the 
second principle is applied. Figure 26 corresponds 
to the third example of the prior art technology 
shown in Figure 5, and additionally contains an MC 
replacement unit 118 for replacing block data using 

45 a motion vector received from the sending side 
when cell-discard is detected. 

Figure 27 is a configurational block diagram of 
the MC replacement unit 118 shown in Figure 26. 
The configuration shown in Figure 27 is the same 

so as that shown in Figure 25 except that a 2-block 
line delay unit 120 and a 1 -block line delay unit 
121 for delaying a motion vector are added such 
that the motion vector can use data received from 
the sending side, and a motion vector detector 122 

55 outputs the motion vector to the replacement con- 
troller 116. 

The flowchart of the data replacement process 
performed by the replacement controller 116 is the 
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same as that shown in Figure 20. When block data 
are replaced using an average value of motion 
vectors for the blocks above and below the target 
block of the replacement, the process of obtaining 
an average value of motion vectors is the same as 
that shown in Figure 21. However, the motion vec- 
tor itself should be received from the sending side. 
Therefore, the processed in steps S 63 and S 66 
are not required. Figure 28 is a flowchart for ex- 
plaining a motion vector average vajue calculating 
process. 

As described above, the present invention re- 
duces the serious deterioration of image quality 
caused by cell-discard by, on a sending side, per- 
forming a compulsory intra-frame encoding on the 
next frame after sending data having a compara- 
tively large difference and encoded by the inter- 
frame encoding system, and by, on a receiving 
side, replacing the data in a discarded block with a 
motion vector after detecting it. Thus, the present 
invention can limit the deterioration of the image 
quality to a visually acceptable level, and greatly 
improve the quality of images processed by an 
image transmission system. 

Reference signs in the claims are intended for 
better understanding and shall not limit the scope. 

Claims 

1. An image encoding and transmitting system 
for dividing inputted frame image data into a 
plurality of blocks comprising picture elements, 
encoding them in block units, and transmitting 
said encoded data in packet units, said system 
comprising on its sending side: 

intra-frame/inter-frame determining means 
(22) for receiving the first difference data as 
the difference between inputted frame data 
and data in the preceding frame or the dif- 
ference between inputted frame data and data 
obtained by multiplying said data in the pre- 
ceding frame by a constant a, and selecting 
and outputting either of the intra-frame data as 
the inputted frame data or the inter-frame data 
as the first difference data, and 

difference collating means (23) for receiv- 
ing inputted frame data and the second dif- 
ference data between said inputted frame data 
and the data in one frame before the present 
frame, and having said inter-frame/intra-frame 
determiner (22) compulsorily output the intra- 
frame data at the data input of the next frame 
for the same block position when more than 
one piece of data in each block forming the 
second difference data are larger than a pre- 
determined threshold, wherein 

the deterioration of image quality of frame 
data having a large difference can be pre- 



vented. 

2. The image encoding and transmitting system 
according to Claim 1, wherein 

5 said* intra-frame/inter-frame determining 

means (22) comprises: 

an intra-frame/inter-frame determiner (12), 
said difference collating means comprises: 
a difference collator (31) for determining 
w whether or not the value of one or more data in 

blocks involved in said second difference data 
is larger than a predetermined threshold, and 

a one-frame delay unit (11) for delaying 
said determination result by 1 frame when said 
;s difference collator (31) determines that one or 

more data in said block is larger than said 
threshold to compulsorily output said data in 
said frame to said intra-frame/inter-frame de- 
terminer (12), 

20 said image transmitting system further 

comprises on its sending side: 

a discrete cosine transform unit (13) for 
performing a discrete cosine transform on in- 
putted image data, 

25 a first subtracter for obtaining the differ- 

ence between said input frame data as an 
output of said discrete cosine transform and 
data obtained by multiplying the data in the 
preceding frame by a leak coefficient a as said 

30 constant to output the result as said first dif- 

ference data to said intra-frame/inter-frame de- 
terminer (12), 

a quantizer (14) for quantizing an output 
from said intra-frame/inter-frame determiner 

35 (12), 

an adder (16) for outputting the sum of an 
output from said quantizer (14) and data ob- 
tained by multiplying the data in the preceding 
frame by a leak coefficient a when said intra- 

40 frame/inter-frame determiner (12) outputs inter- 

frame data, and outputting an output from said 
quantizer (14) as is when it outputs intra-frame 
data, 

a frame memory (9) for storing an output 

45 from said adder (16). 

a leak coefficient unit (10) for multiplying 
an output from said frame memory (9) by a 
leak coefficient a to output the result to said 
first subtracter (1 1) and said adder (16), and 

so a second subtracter (30) for obtaining the 

difference between an output of said discrete 
cosine transform (13) and an output from said 
frame memory (9) to output it as said second 
difference data to said difference collator (31). 

55 

3. The image encoding and transmitting system 
according to Claim 2, wherein 

said difference collator (31) comprises: 
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comparators (33) as many as the number 
of blocks in one frame for comparing the ab- 
solute value of said first difference data for 
each piece of data with a predetermined 
threshold, and outputting "0 M when said ab- 5 
solute value of said first difference data is 
greater than said threshold and "1" when it is 
smaller than said threshold, and 

an AND circuit (34) for receiving an output 
of each of said comparators (33), wherein w 

said one-frame delay unit comprises a 
first-in-first-out memory (35) for delaying by 
one frame an output from said AND circuit 
(34). 

15 

4. The image encoding and transmitting system 
according to Claim 3, wherein 

said intra-frame/inter-frame determiner 

(12) comprises: 

a first sum-of-squares unit (36a) for receiv- 20 
ing an output from said first subtracter (11) and 
calculating a sum of squares of data in each 
block for said first difference data, 

a second sum-of-squares unit (36b) for 
receiving inputted frame data as an output 25 
from said discrete cosine transform unit (13), 
and calculating a sum of squares of data in 
each block for said input frame data, 

a comparator (37) for comparing an output 
from said first sum-of-squares (36a) with an 30 
output from said second sum-of-squares (36b), 
and outputting "0" when said output from said 
first sum-of-squares (36a) is larger than said 
output from said second sum-of-squares (36b), 
and "1" when it is smaller than said output 35 
from said second sum-of-squares, 

an AND circuit (38) for receiving an output 
from said comparator (37) and an output from 
said first-in-first-out memory (35), 

a first delay unit (39a) for delaying an 40 
output from said first subtracter (11) to the 
time point when a comparison result is output- 
ted by said comparator (37). 

a second delay unit (39b) for delaying an 
output from said discrete cosine transform unit 45 

(13) by the delay time of said first delay unit 
(39a), and 

a selector (40) for selecting and outputting 
an output from said first delay unit (39a) when 
said AND circuit (38) outputs "1 ", and an out- so 
put from said second delay unit (39b) when 
said AND circuit (38) outputs "0". 

5. An image encoding and transmitting system 

for dividing inputted frame image data into a 55 
plurality of blocks comprising picture elements, 
encoding them in block units, and transmitting 
said encoded data in packet units, comprising 



on its receiving side: 

picture element domain frame data storing 
means (24) for storing received image data in 
the picture element domain, and 

motion vector detecting means (25) for de- 
tecting a motion vector using the data in the 
blocks surrounding the block whose encoded 
data were not received due to packet-discard 
among the frame data stored in the picture 
element domain frame data storage (24) when 
a packet-discard detection signal is inputted, 
and replacing using said motion vector the 
data in the blocks whose encoded data are not 
received. 

6. The image encoding and transmitting system 
according to Claim 5, wherein 

a sequence number is assigned to said 
packet on the sending side to transmit said 
packet, and 

packet-discard is detected on the receiving 
side according to said sequence numbers. 

7. The image encoding and transmitting system 
according to Claim 5, 

said image domain frame data storage 
(24) comprises a frame memory (41), 

said motion vector detecting means (25) 
comprises a motion compensation MC replace- 
ment unit (42), 

said image encoding and transmitting sys- 
tem further comprises on its receiving side: - 

an inverse discrete cosine transform unit 
(17) for performing an inverse discrete cosine 
transform on received data, 

an adder (20) for obtaining the sum of the 
data in the present frame as an output from 
said inverse discrete cosine transform unit (17) 
and the data obtained by multiplying the data 
in the preceding frame by a leak coefficient a 
to output it to said frame memory (41), and 

a leak coefficient unit (19) for multiplying 
an output from said frame memory by a leak 
coefficient # to output the result to said adder 
(20). 

8. The image encoding and transmitting system 
according to Claim 7, further comprising before 
said inverse discrete cosine transform unit 
(17): 

a header remover (103) for removing a 
header from a received packet, 

a separator (104) for separating a packet 
whose header is removed into a sequence 
number and encoded data, 

a counter (105) to be stepped up when 
said packet is received, 

a comparator (106) for comparing a count 
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value outputted from said counter (105) with a 
sequence number outputted by said separator 
(104), and outputting said detection signal 
when the discard of a packet is detected, 

a memory (107) for storing encoded data 5 
outputted by said separator (104), and 

a read, controller (108) for outputting the 
content stored in said memory (107) when said 
comparator (106) does not detect the discard 
of a packet, and outputting dummy data "0" as w 
encoded data to said inverse discrete cosine 
transform unit (17). 

9. The image encoding and transmitting system 
according to Claim 7, wherein 75 

said motion compensation replacement 
unit (42) comprises: . 

a 2-block-line delay unit (110) and a*1- 
block-line delay unit (111) for delaying said 
packet discard detection signal by 2 block 20 
lines and 1 block line, 

a B(i,j-8) discard detector (112) for detect- 
ing the discard of a block above the target 
block to be replaced according to an output 
from said 2-block-line delay unit (110), 25 

a B(i,j) discard detector (113) for detecting 
the discard of said target block to be replaced 
according to an output from said 1 -block-line 
delay unit (111), 

a B(i,j+8) discard detector (114) for de- 30 
tecting the discard of a block below said target 
block to be replaced according to an input 
from said packet discard detection signal, 

a motion vector detector (115) for detect- 
ing a motion vector using the outputs from said 35 
three discard detectors (112. 113, 114) and the 
content stored in said frame memory (41), and 

a replacement controller (116) for replac- 
ing data using said detected motion vector. 

40 

10. An image encoding and transmitting system 
according to Claim 5, further comprising on its 
receiving side: 

coefficient domain frame data storage 
means for storing image data in the frequency 45 
coefficient domain, wherein 

the block data in the preceding frame 
stored in said coefficient domain frame data 
storing means (24) are transformed to the pic- 
ture element domain, and so 

said transformed data are stored in said 
picture element domain frame data storage 
means. 

11.. The image encoding and transmitting system 55 
according to Claim 10, 

said coefficient domain claim data storage 
comprises a first frame memory (18), 
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said picture domain frame data storage 
(24) comprises a second frame memory (43), 

said motion vector detecting means (25) 
comprises a motion compensation (MC) re- 
placement unit (44), 

said image encoding and transmitting sys- 
tem further comprising on its receiving side: 

an adder (20) for obtaining the sum of the 
received data in the present frame and the 
data obtained by multiplying the data in the 
preceding frame by a leak coefficient a when 
packet discard is not detected, and outputting 
said data in the preceding frame as is to said 
first frame memory (18) when packet discard is 
detected, 

a leak coefficient unit (19) for outputting to 
said adder (20) a value obtained by multiplying 
the output from said first frame memory (18) 
by a leak coefficient a when packet discard is 
not detected, or outputting an output from said 
first frame memory (18) as is to said adder 
(20) when packet discard is detected, and 

an inverse discrete cosine transform unit 
(17) for performing an inverse discrete cosine 
transform on an output from said adder (20) 
and output the result to said second frame 
memory (43). 

12. The image encoding and transmitting system 
according to Claim 11, further comprising be- 
fore, said adder (20): 

a header remover (103) for removing a 
header from a received packet, 

a separator (104) for separating a packet 
whose header is removed into a sequence 
number and encoded data, 

a counter (105) to be stepped up when 
said packet is received, 

a comparator (106) for comparing a count 
value outputted from said counter (105) with a 
sequence number outputted by said separator 
(104), and outputting said detection signal 
when the discard of a packet is detected, 

a memory (107) for storing encoded data 
outputted by said separator (104), and 

a read controller (108) for outputting the 
content stored in said (107) memory when said 
comparator (106) does not detect the discard 
of a packet, and outputting dummy data M 0" as 
encoded data to sakl adder (20). 

13. The image encoding and transmitting system 
according to Claim 1 1, wherein 

said motion compensation replacement 
unit (44) comprises: 

a 2-block-line delay unit (110) and a 1- 
block-tine delay unit (111) for delaying said 
packet discard detection signal by 2 block 
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lines and 1 block line, 

a B(i J-S) discard detector (112) for detect- 
ing the discard of a block above the target 
block to be replaced according to an output 
from said 2-block-line delay unit (1 10), 5 

a B(i t j) discard detector (1 13) for detecting 
the discard of said target block to be replaced 
according to an output from said 1 -block-line - 
delay unit (111), 

a B(i,j + 8) discard detector (14) for detect- w 
ing the discard of a block below said target 
block to be replaced according to an input 
from said packet discard detection signal, 

a motion vector detector (115) for detect- 
ing a motion vector using the outputs from said ;s 
three discard detectors (112, 113, 114) and the 
content stored in said frame memory (41), and 

a replacement controller (116) for replac- 
ing data using said detected motion vector. 

20 

14. The image encoding and transmitting system 
according to Claim 5, wherein 

in a data replacement process using said 
motion vector, determination is made as to 
whether or not block B(iJ) having its upper left 25 
corner point (i,j) is discarded, 

if no, the process flow is terminated, 

if yes, the variable y indicating the vertical 
coordinate is set to "0", 

the variable x indicating the horizontal co- 30 
ordinate is set to "0", 

the horizontal coordinate (i+x) and the 
vertical coordinate (j + y) in the present frame 
are replaced with the horizontal coordinate 
(i+x+ x component of the motion vector) and 35 
the vertical coordinate (j + v+ y component of 
the motion vector) in the preceding frame, 

the value of the variable x indicating the 
horizontal coordinate is stepped up. 

determination is made as to whether or not 40 
"x" has reached 8, 

if "x" has not reached 8, the processes 
are repeated from the process in which said 
present frame data are replaced with the pre- 
ceding frame data, 45 

if "x" has reached 8, the value of the 
variable y is stepped up, 

determination is made as to whether or not 
"y" has reached 8, 

if M y" has not reached 8, the processes so 
are repeated from the process in which the 
variable x indicating said horizontal coordinate 
is set to "0", and 

if M y" has reached 8, the process flow is 
terminated. 55 

15. The image encoding and transmitting system 
according to Claim 14, wherein 



an average value of motion vectors of one 
or more blocks not affected by packet-discard 
among blocks surrounding the block whose 
encoded data are not received is used as a 
motion vector in replacing said block data. 

16. The image encoding and transmitting system 
according to Claim 15, wherein 

in a moving vector average value calculat- 
ing process, determination is made as to 
whether or not block B(i,j) having its upper left 
corner point (i.j) is discarded, 

if no, the process flow is terminated, 

if yes, the variable blkct indicating the 
number of significant blocks for use in detect- 
ing a motion vector is set to "0", 

determination is made as to whether or not 
block B(i,j-8) above said discarded block B(ij) 
is discarded. 

if said block B(i,j-8) is not discarded, the 
vector 1x as the x component vector and the 
vector 1y as the y component of said motion 
vector for said block B(i,j-8) are detected, 

the value of the variable blkct as said 
number of significant blocks is stepped up, 
said vector 1x is set as a vector x as the x 
component of a motion vector to be processed 
in said average value calculation, and said 
vector 1 y is set as a vector y as the y compo- 
nent of a motion vector to be processed in 
said average value calculation, 

if said block B(iJ-8) is discarded, no ac- 
tions are taken, 

determination is made whether of not 
block B(i,j + 8) below said discarded block B(iJ) 
is discarded, 

if said block B(i,j + 8) is not discarded, a 
vector 2x as the x component of the motion 
vector for said block B(ij+8) and a motion 
vector 2y as the y component are detected, 

a value of the variable blkct indicating said 
number of significant blocks is stepped up, 
said vector 2x is added to the x component of 
the motion vector to be processed in said 
average value calculation to obtain the x com- 
ponent of a new motion vector, said vector 2y 
is added to the y component of the motion 
vector to be processed in said average value 
calculation to obtain the y component of a new 
motion vector, 

if said block B(i,j + 8) is discarded, no ac- 
tions are taken, 

determination is made as to whether or not 
said variable blkct equals "0", 

if said variable blkct is "0", the x and y 
components of the motion vector are set to "0" 
and the process flow is terminated, and 

if said variable blkct is not "0", the x 
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component of the motion vector to be pro- 
cessed in said average value calculation is 
divided by the variable blkct. and the result is 
assigned as the x component of the motion 
vector, the y component of the motion vector 
to be processed in said average value calcula- 
tion is divided by the variable blkct. and the 
result is assigned as the y component of the 
motion vector, and the process flow is termi- 
nated. 

17. The image encoding and transmitting system 
according to Claim 16, wherein 

in a motion vector detecting process for 
said undiscarded block, a large value is set as 
the initial value of sum 1 which is a sum of 
absolute values, for all data in a . block, of. 
difference data between received data in a 
block whose motion vector is to be detected in 
the present frame and its corresponding data 
in the preceding frame, 

the initial value of the variable vecy in- 
dicating the y component of a motion vector is 
set to "-4", 

the initial value of the variable vecx indicat- 
ing the x component of a motion vector is set 
to "-4", 

the sum "sum 2" of the absolute values of 
the differences between each piece of the 
present frame data for the point (i +x, j + y) and 
the preceding frame data for the point 
(i + x + vecx, j + y + vecy) for all blocks is ob- 
tained, 

determination is made as to whether or not 
the value of said sum 2 is smaller than the 
value of said sum 1, 

if yes, vecx is set as the x component of a 
motion vector, vecy as its y component, and 
the value of sum 2 is substituted for sum 1, 

if no, no actions are taken, 

the value of the variable vecx is stepped 

up. 

determination is made as to whether or not 
the value of vecx has reached "5", 

if not, the processes are repeated from the 
process in which said sum 2 is obtained, 

if yes, the value of the variable vecy is 
stepped up, 

determination is made as to whether of not 
the value of vecy has reached "5", 

if not, the processes are repeated from the 
process in which the value of said variable 
vecx is set to the initial value -4, 

if yes, the process flow is terminated. 

18. The image encoding and transmitting system 
according to Claim 14, wherein 

a motion vector is detected in a block 



group which contains one or more blocks not 
affected by packet discard among blocks sur- 
rounding a block whose encoded data are not 
received, and 

5 said motion vector is used for replacing 

said block data. 

19. The image encoding and transmitting system 
according to Claim 18. wherein 

10 in a motion vector calculating process for 

said block group, determination is made as to 
whether or not block B(i.j) having its upper left 
corner point (i,j) is discarded. 

if no, the process flow is terminated, 

75 if yes, determination is made as to wheth- 

er or not block B(i.j-8) above said block B(i,j) is 
discarded, 

if yes, the minimum value mini of the 
vertical coordinate of said group processed in 
20 calculating said vector is set to "j". 

if not, said value minj is set to "j-8", 
determination is made as to whether or not 
block B(i,j + 8) below said discarded block B- 

(ij). 

25 if yes, the maximum value maxj of the 

vertical coordinate of said group processed in 
calculating said vector is set to "j + 7", 
if not said value maxj is set to "j + 15", 
determination is made as to whether or not 

30 said value "minj" equals "j" and said value 

"maxj" equals "j + 7", 

if yes, the x and y components of said 
motion vector are set to "0", and the process 
flow is terminated, and 

35 if not, a motion vector is detected in a 

block group which does not contain a dis- 
carded block between said minimum value 
"minj" and said maximum value "maxj" of the 
vertical coordinate, and the process flow is 

40 terminated. 

20. The image encoding and transmitting system 
according to Claim 19. wherein 

in a motion vector detecting process for 

45 said undiscarded block, a targe value is set as 

the initial value of sum 1 which is the minimum 
sum of absolute values of difference data be- 
tween received data in a block whose motion 
vector is to be detected in the present frame 

so and its corresponding data in the preceding 

frame, 

the initial value of the variable vecy in- 
dicating the y component of a motion vector is 
set to "-4". 

55 the initial value of the variable vecx indicat- 

ing the x component of a motion vector is set 
to "-4", 

the sum "sum 2" of the absolute values of 
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the differences between each piece of the 
present frame data for the point (i + x, j + y) and 
the preceding frame data for the point 
(i+x + vecx, j + y + vecy) for all blocks in said 
group is obtained. s 

determination is made as to whether or not 
the value of said sum 2 is smaller than the 
value of said sum 1 , 

if yes, vecx is set as the x component of a 
motion vector, vecy as its y component, and 10 
the value of sum 2 is substituted for sum 1 , 

if no, no actions are taken, 

the value of the variable vecx is stepped 

up, 

determination is made as to whether or not is 
the value of vecx has reached "5", 

if not, the processes are repeated from the 
process in which said sum 2 is obtained, 

if yes, the value of the variable vecy is 
stepped up, 20 

determination is made as to whether of not 
the value of vecy has reached "5", 

if not, the processes are repeated from the 
process in which the value of said variable 
vecx is set to the initial value -4, 25 

if yes, the process flow is terminated. 

21. An image encoding and transmitting system 
used in an image transmission system for di- 
viding inputted frame image data into blocks 30 
comprising a plurality of picture elements, en- 
coding said data in block units, and transmit- 
ting said encoded data in packet units, said 
system comprising: 

a motion vector detector (100) on a send- 55 
ing side of said image transmission system for 
detecting a motion vector for each of said 
block and sending said motion vector to a 
receiving side, 

an inverse discrete cosine transform unit 40 
(17) on said receiving side for performing an 
inverse discrete cosine transform on received 
data, 

an adder (20) for adding the present frame 
data as an output from said discrete cosine 45 
transform unit (17) to data obtained by mul- 
tiplying the delay result obtained, by delaying 
data in the preceding frame according to a 
motion vector received from said sending side 
by a leak coefficient a, so 

a frame memory (18) for storing an output 
of said adder (20), 

a motion compensation replacement unit 
(MC)(118) for replacing the data for a block 
whose encoded data are not received due to 55 
packet-discard using a motion vector received 
from said sending side and the content shored 
in said frame memory (18) when a packet 



discard detection signal is inputted, 

a variable delay unit (102) for delaying an 
output from said frame memory (18) according 
to a motion vector received from said sending 
side, 

a leak coefficient unit (19) for multiplying 
an output from said variable delay unit (102) 
by a leak coefficient a and outputting the prod- 
uct to said adder (20). 

22. The image encoding and transmitting system 
according to Claim 21, further comprising be- 
fore said inverse discrete cosine transform unit 
(17): 

a header remover (103) for removing a 
header from a received packet, 

.. a separator (104) for separating a packet 
whose header is removed into a sequence 
number and encoded data, 

a counter (105) to be stepped up when 
said packet is received, 

a comparator (106) for comparing a count 
value outputted from said counter (105) with a 
sequence number outputted by said separator 
(104), and outputting said detection signal 
when the discard of a packet is detected, 

a memory (107) for storing encoded data 
outputted by said separator (104), and 

a read controller (108) for outputting the 
content stored in said memory (107) when said 
comparator (106) does not detect the discard 
of a packet, and outputting dummy data "0" as 
encoded data to said inverse discrete cosine 
transform unit (17). 

23. The image encoding and transmitting system 
according to Claim 21, wherein 

said motion compensation replacement 
unit (1 18) comprises: 

a 2-bIock-line delay unit (110) and a 1- 
block-line delay unit (111) for delaying said 
packet discard detection signal by 2 block 
lines and 1 block line, 

a B(ij-8) discard detector (112) for detect- 
ing the discard of a block above the target 
block to be replaced according . to an output 
from said 2-block-line delay unit (10), 

a B(i,j) discard detector (113) for detecting 
the discard of said target block to be replaced 
according to an output from said 1 -block-line 
deiay unit (111), 

a B(i,j + 8) discard detector (114) for de- 
tecting the discard of a block below said target 
block to be replaced according to an input 
from said cell-discard detection signal, 

1 -block-line and 2-block-Iine delay units 
(120, 121) for delaying a motion vector re- 
ceived from said sending side by 1 block and 
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2 blocks respectively. 

a motion vector detector (122) for detect- 
ing a motion vector using the outputs from said 
three discard detectors (112. 113. 114) and 
two delay units (120. 121). and the present s 
value of a motion vector received from said 
sending side, and 

a replacement controller (116) for replac- 
. ing data using said detected motion vector. 

70 

24. The image encoding and transmitting system 
according to Claim 21, wherein 

an average value of motion vectors of one 
or more blocks not affected by packet-discard 
among blocks surrounding the block whose is 
encoded data are not received is used as a 
motion vector in replacing said block data, 

25. The image encoding and transmitting system 
according to Claim 24, wherein 20 

in a moving vector average value calculat- 
ing process, determination is made as to 
whether or not block B(iJ) having its upper left 
corner point (i.j) is discarded, 

if no, the process flow is terminated, 25 

if yes, the variable blkct indicating the 
number of significant blocks for use in detect- 
ing a motion vector is set to "0'\ 

determination is made as to whether or not 
block B(i,j-8) above said discarded block B(ij) 30 
is discarded, 

if said block B(i,j-8) is not discarded, the x 
component of said motion vector for said block 
B(i.j-S) received from the transmitting side is 
set to vector 1x, and the y component of said 35 
motion vector is set to vector 1y, 

the value of the variable blkct as said 
number of significant blocks is stepped up, 
said vector 1x is set as a vector x as the x 
component of a motion vector to be processed 40 
in said average value calculation, and said 
vector 1 y is set as a vector y as the y compo- 
nent of a motion vector to be processed in 
said average value calculation, 

if said block B(i,j-8) is discarded, no ac- 45 
tions are taken, 

determination is made whether of not 
block B(i,j + 8) below said discarded block B(i,j) 
is discarded, 

if said block B(i.j + 8) is not discarded, the 50 
x component of the motion vector for said 
block B(i,j + 8) received from the sending side 
is set to vector 2x, and a the y component of 
the motion vector is set to vector 2y, 

a value of the variable blkct indicating said 55 
number of significant blocks is stepped up, 
said vector 2x is added to the x component of 
the motion vector to be processed in said 



average value calculation to obtain the x com- 
ponent of a new motion vector, said vector 2y 
is added to the y component of the motion 
vector to be processed in said average value 
calculation to obtain the y component of a new 
motion vector, 

if said block B(i,j + 8) is discarded, no ac- 
tions are taken. 

determination is made as to whether or not 
said variable blkct equals "0". 

if said variable blkct is "0". the x and y 
components of the motion vector are set to "0" 
and the process flow is terminated, and 

if said variable blkct is not rt 0", the x 
component of the motion vector to be pro- 
cessed in said average value calculation is 
divided by the variable blkct, and the result is 
assigned as the x component of the motion 
vector, the y component of the motion vector 
to be processed in said average value calcula- 
tion is divided by the variable blkct, and the 
result is assigned as the y component of the 
motion vector, and the process flow is termi- 
nated. 
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